Search for: All records

The recent progress of AI can be largely attributed to large language models (LLMs). However, their escalating memory requirements introduce challenges for machine learning (ML) researchers and engineers. Addressing this requires developers to partition a large model to distribute it across multiple GPUs or TPUs. This necessitates considerable coding and intricate configuration efforts with existing model parallel tools, such as Megatron-LM, DeepSpeed, and Alpa. These tools require users’ expertise in machine learning systems (MLSys), creating a bottleneck in LLM development, particularly for developers without MLSys background. In this work, we present RedCoast (Redco), a lightweight and user-friendly tool crafted to automate distributed training and inference for LLMs, as well as to simplify ML pipeline development. The design of Redco emphasizes two key aspects. Firstly, to automate model parallelism, our study identifies two straightforward rules to generate tensor parallel strategies for any given LLM. Integrating these rules into Redco facilitates effortless distributed LLM training and inference, eliminating the need of additional coding or complex configurations. We demonstrate the effectiveness by applying Redco on a set of LLM architectures, such as GPT-J, LLaMA, T5, and OPT, up to the size of 66B. Secondly, we propose a mechanism that allows for the customization of diverse ML pipelines through the definition of merely three functions, avoiding redundant and formulaic code like multi-host related processing. This mechanism proves adaptable across a spectrum of ML algorithms, from foundational language modeling to complex algorithms like meta-learning and reinforcement learning. As a result, Redco implementations exhibit significantly fewer lines of code compared to their official counterparts. RedCoast (Redco) has been released under Apache 2.0 license at https://github.com/tanyuqian/redco. 

The diphosphine complexes cis - or trans -PtCl 2 (P((CH 2 ) n ) 3 P) ( n = b/12, c/14, d/16, e/18) are demetalated by MCX nucleophiles to give the title compounds (P((CH 2 ) n ) 3 )P (3b–e, 91–71%). These “empty cages” react with PdCl 2 or PtCl 2 sources to afford trans -MCl 2 (P((CH 2 ) n ) 3 P). Low temperature 31 P NMR spectra of 3b and c show two rapidly equilibrating species (3b, 86 : 14; 3c, 97 : 3), assigned based upon computational data to in , in (major) and out , out isomers. These interconvert by homeomorphic isomerizations, akin to turning articles of clothing inside out (3b/c: Δ H ‡ 7.3/8.2 kcal mol −1 , Δ S ‡ −19.4/−11.8 eu, minor to major). At 150 °C, 3b, c, e epimerize to (60–51) : (40–49) mixtures of ( in , in / out , out ) :  in , out isomers, which are separated via the bis(borane) adducts 3b, c, e·2BH 3 . The configurational stabilities of in , out -3b, c, e preclude phosphorus inversion in the interconversion of in , in and out , out isomers. Low temperature 31 P NMR spectra of in , out -3b, c reveal degenerate in , out / out , in homeomorphic isomerizations (Δ G ‡Tc 12.1, 8.5 kcal mol −1 ). When ( in , in / out , out )-3b, c, e are crystallized, out , out isomers are obtained, despite the preference for in , in isomers in solution. The lattice structures are analyzed, and the D 3 symmetry of out , out -3c enables a particularly favorable packing motif. Similarly, ( in , in / out , out )-3c, e·2BH 3 crystallize in out , out conformations, the former with a cycloalkane solvent guest inside. 

The gyroscope like dichloride complexes trans -Pt(Cl) 2 (P((CH 2 ) n ) 3 P) ( trans -2; n = c, 14; e, 18; g, 22) and MeLi (2 equiv.) react to yield the parachute like dimethyl complexes cis -Pt(Me) 2 (P((CH 2 ) n ) 3 P) ( cis -4c,e,g, 70–91%). HCl (1 equiv.) and cis -4c react to give cis -Pt(Cl)(Me)(P((CH 2 ) 14 ) 3 P) ( cis -5c, 83%), which upon stirring with silica gel or crystallization affords trans -5c (89%). Similar reactions of HCl and cis -4e,g give cis / trans -5e,g mixtures that upon stirring with silica gel yield trans -5e,g. A parallel sequence with trans -2c/EtLi gives cis -Pt(Et) 2 (P((CH 2 ) 14 ) 3 P) ( cis -6c, 85%) but subsequent reaction with HCl affords trans -Pt(Cl)(Et)(P((CH 2 ) 14 ) 3 P) ( trans -7c, 45%) directly. When previously reported cis -Pt(Ph) 2 (P((CH 2 ) 14 ) 3 P) is treated with HCl (1 equiv.), cis - and trans -Pt(Cl)(Ph)(P((CH 2 ) 14 ) 3 P) are isolated (44%, 29%), with the former converting to the latter at 100 °C. Reactions of trans -5c and LiBr or NaI afford the halide complexes trans -Pt(X)(Me)(P((CH 2 ) 14 ) 3 P) ( trans -9c, 88%; trans -10c, 87%). Thermolyses and DFT calculations that include acyclic model compounds establish trans > cis stabilities for all except the dialkyl complexes, for which energies can be closely spaced. The σ donor strengths of the non-phosphine ligands are assigned key roles in the trends. The crystal structures of cis -4c, trans -5c, trans -7c, and trans -10c are determined and analyzed together with the computed structures. 

High-voltage lithium-metal batteries (LMBs) with LiCoO 2 (LCO) as the cathode have high volumetric and gravimetric energy densities. However, it remains a challenge for stable cycling of LCO >4.5 V Li . Here we demonstrate that a rationally designed sulfonamide-based electrolyte can greatly improve the cycling stability at high voltages up to 4.7 V Li by stabilizing the electrode–electrolyte interfaces (EEIs) on both the Li-metal anode (LMA) and high-voltage LCO cathode. With the sulfonamide-based electrolyte, commercial LCO cathodes retain 89% and 85% of their capacities after 200 and 100 cycles under high charging voltages of 4.55 V Li and 4.6 V Li , respectively, significantly outperforming traditional carbonate-based electrolytes. The surface degradation, impedance growth, and detrimental side reactions in terms of gas evolution and Co dissolution are well suppressed. Our work demonstrates a promising strategy for designing new electrolytes to realize high-energy Li||LCO batteries.